Image capture system with image conversion mechanism and method of operation thereof

ABSTRACT

A three dimensional image capture system includes: an image capture device configured to generate video data; a lens, coupled to the image capture device, configured to focus a left image and a right image; a microprism array, coupled to the lens, configured to horizontally deflect the left image and the right image; and an image processing unit, coupled to the image capture device, configured to calculate a depthmap from the left image and the right image in the video data, rendered by the microprism array.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims the benefit of U.S. Provisional PatentApplication Ser. No. 61/702,571 filed Sep. 18, 2012, and the subjectmatter thereof is incorporated herein by reference thereto.

TECHNICAL FIELD

An embodiment of the present invention relates generally to imagecapture system, and more particularly to a system for image conversion.

BACKGROUND

Modern consumer and industrial electronics, especially devices such asgraphical display systems, televisions, projectors, cellular phones,portable digital assistants, and combination devices, are providingincreasing levels of functionality to support modern life includingthree-dimensional (3D) display services. Research and development in theexisting technologies can take a myriad of different directions.

3D image capturing generally requires two image capture modules: a firstimage capture module imitates the human left eye; and a second imagecapture module imitates the human right eye. The combination of thefirst image and the second image can present very difficult technicalissues.

In conventional techniques, the first and second image capture modules,assembled in a portable electronic device with 3D, are spaced apart by afixed distance. When a subject to be captured is very close to thedevice, the image difference between a first image captured by the firstimage capture module and a second image captured by the second imagecapture module may be too significant to form a 3D image.

If the first image and the second image are not properly combined, theresultant image can look unnatural or present an unnerving effect on theviewer. In many cases having an incorrect blending of the first imageand the second image can result in a shadow image that can give a viewera headache when it is observed.

Thus, a need still remains for a three dimensional image capture systemwith image conversion mechanism to display three-dimensional images. Inview of the ever-increasing commercial competitive pressures, along withgrowing consumer expectations and the diminishing opportunities formeaningful product differentiation in the marketplace, it isincreasingly critical that answers be found to these problems.Additionally, the need to reduce costs, improve efficiencies andperformance, and meet competitive pressures adds an even greater urgencyto the critical necessity for finding answers to these problems.

Solutions to these problems have been long sought but prior developmentshave not taught or suggested any solutions and, thus, solutions to theseproblems have long eluded those skilled in the art.

SUMMARY

An embodiment of the present invention provides a three dimensionalimage capture system, including: an image capture device configured togenerate video data; a lens, coupled to the image capture function,configured to focus a left image and a right image; a microprism array,optically coupled to the lens, configured to horizontally deflect theleft image and the right image; and an image processing unit, coupled tothe image capture function, configured to calculate a depthmap from theleft image and the right image in the video data, rendered by themicroprism array.

An embodiment of the present invention provides a method of operation ofa three dimensional image capture system including: illuminating animage capture function configured to generate video data; focusing aleft image and a right image, through a lens, on the image capturefunction; horizontally deflecting the left image and the right imagefrom a microprism array; and calculating a depthmap from the left imageand the right image in the video data.

Certain embodiments of the invention have other steps or elements inaddition to or in place of those mentioned above. The steps or elementswill become apparent to those skilled in the art from a reading of thefollowing detailed description when taken with reference to theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an architecture diagram of a three dimensional image capturesystem with image conversion mechanism in an embodiment of the presentinvention.

FIG. 2 is an architecture diagram of a three dimensional image capturesystem with image conversion mechanism in an example implementation.

FIG. 3 is an exemplary block diagram of the three dimensional imagecapture system.

FIG. 4 is a hardware block diagram of the three dimensional imagecapture system.

FIG. 5 is a flow chart of a method of operation of a three dimensionalimage capture system for performing a depthmap generation.

FIG. 6 is an example of a video frame as parsed by the correspondencepoints function of FIG. 5.

FIG. 7 is an example of a video frame as manipulated by the computedisparity function of FIG. 5.

FIG. 8 is a control flow of the compute depthmap function.

DETAILED DESCRIPTION

An embodiment of the present invention can provide a three dimensionalimage from a two dimensional camera structure. By applying themicroprism array spaced away from the lens, the left image and the rightimage can be analyzed for forming the three dimensional image of thetarget object. The hardware portion of the three dimensional imagecapture system can predictably produce the left image and the rightimage for processing of the three dimensional image.

An embodiment of the present invention can provide the combination ofthe microprism array and the image capture function can be a lessexpensive and less cumbersome solution that is equivalent to atraditional stereo system consisting of two cameras placed at Oseparated by a baseline distance B given by B=2·u_(z)·tan(δ).

An embodiment of the present invention can provide the fixed value ofthe deviation angle (δ) can provide a standard separation of the leftimage and the right image on the image capture function. The standardseparation of the left image and the right image can aid in theidentification and correlation of matching points for producing thethree dimensional image. This can reduce the circuitry required toproduce the three dimensional image, while providing a detailed threedimensional display.

An embodiment of the present invention can provide a method andapparatus for developing three dimensional images from hardware that wasdeveloped for two dimensional applications. The application of themicroprism array can generate the left image and the right image for asingle instance of the target object and a single instance of the lens.

An embodiment of the present invention can provide the three dimensionalimage based on a single instance of the lens and the microprism array.The video processor can identify correspondence points by thecorrespondence points function, generate the depthmap, and adjust it tomaintain coherence from frame to frame by the temporal coherencecorrection function. The three dimensional image output from thetemporal coherence correction function can be a single frame or a videostream of frames with a three dimensional image of the target object.

An embodiment of the present invention can provide the search for thecorresponding block which can be performed only in the row m because themicroprism array only displaces the left image and the right imagehorizontally due to the prism vertex of the N^(th) microprism and theN+1^(st) microprism being vertically aligned and parallel. Any featureidentified in the left image will have a corresponding feature withinthe same row for the right image. This allows the video processor tominimize the search time for the correspondence point.

The following embodiments are described in sufficient detail to enablethose skilled in the art to make and use the invention. It is to beunderstood that other embodiments would be evident based on the presentdisclosure, and that system, process, or mechanical changes may be madewithout departing from the scope of an embodiment of the presentinvention.

In the following description, numerous specific details are given toprovide a thorough understanding of the invention. However, it will beapparent that the invention may be practiced without these specificdetails. In order to avoid obscuring an embodiment of the presentinvention, some well-known circuits, system configurations, and processsteps are not disclosed in detail.

The drawings showing embodiments of the system are semi-diagrammatic,and not to scale and, particularly, some of the dimensions are for theclarity of presentation and are shown exaggerated in the drawingfigures. Similarly, although the views in the drawings for ease ofdescription generally show similar orientations, this depiction in thefigures is arbitrary for the most part. Generally, the invention can beoperated in any orientation. The embodiments have been numbered firstembodiment, second embodiment, etc. as a matter of descriptiveconvenience and are not intended to have any other significance orprovide limitations for an embodiment of the present invention.

The term “uniform triangular prism” is, for example, a triangular prismhaving equal base angles. The term “module” referred to herein caninclude software, hardware, or a combination thereof in an embodiment ofthe present invention in accordance with the context in which the termis used. For example, the software can be machine code, firmware,embedded code, and application software. Also for example, the hardwarecan be circuitry, processor, computer, integrated circuit, integratedcircuit cores, a pressure sensor, an inertial sensor, amicroelectromechanical system (MEMS), passive devices, or a combinationthereof.

Referring now to FIG. 1, therein is shown an architecture diagram of athree dimensional image capture system 100 with image conversionmechanism in a first embodiment of the present invention. Thearchitecture diagram of the three dimensional image capture system 100includes a lens 102 having an optical axis 104. An image capturefunction 106, such as a charge coupled device (CCD) for detecting theoutput of the lens 102, can be positioned a focal distance (f_(x)) 108from the lens 102. The image capture function 106 can be positionedbehind the lens 102 parallel to the base plane P of a microprism array110. The image capture function 106 can be an image capture device 106or a hardware accelerator including the image capture device 106 withintegrated processor support for executing process software. Themicroprism array 110, such as multiple uniform triangular prismsvertically aligned, can be positioned an array distance (U_(Z)) 112 fromthe lens 102 opposite the focal distance (f_(z)) 108. The microprismarray 110 can include an array of the uniform triangular prism havingequal base angles. The functions described can include hardware forproviding operational support, for example, hardware circuitry,processor, computer, integrated circuit, integrated circuit cores,active devices, passive devices, or a combination thereof.

A target object 114 can be positioned a picture depth (z_(p)) 116 beyondthe microprism array 110. The picture depth (z_(p)) 116 can be thedistance from the microprism array 110 to the target object 114. A leftvirtual object 118 and a right virtual object 120 can be created by themicroprism array 110. The left virtual object 118 and a right virtualobject 120 can be perceived to be spaced apart by a virtual objectseparation distance (D_(px)) 122. It is understood that the left virtualobject 118 and a right virtual object 120 are an artifact of themicroprism array 110.

The light reflecting from the target object 114 can actually follow thepath from the left virtual object 118 and the right virtual object 120through the lens 102 to illuminate the image capture function 106 inorder to be detected as a left image 124 and a right image 126. Theimage capture function 106 can be optically coupled to the lens 102 andconfigured to record the left image 124 and the right image 126. Theoptical coupling of the lens 102 and the image capture function 106 canprovide the left image 124 and the right image 126 focused on the imagecapture function 106 through the lens 102. The left image 124 and theright image 126, on the image capture function 106, can be spaced apartby an image separation distance (d_(px)) 128.

The relation between the virtual object separation distance 122 alongthe axis between these two virtual objects and the image separationdistance 128 between their images is given by:

$\begin{matrix}{d_{px} = {\frac{z_{i}}{z_{p} + u_{z}} \cdot D_{px}}} & {{Equation}\mspace{14mu} 1}\end{matrix}$

Where is is calculated by:D _(px)=2·z _(p)·tan(δ)  Equation 2

and the deviation angle (δ) is calculated by:

$\begin{matrix}{\delta_{m} = {{2\;{\sin^{- 1}\left( {n\;{\sin\left( \frac{\alpha}{2} \right)}} \right)}} - \alpha}} & {{Equation}\mspace{14mu} 3}\end{matrix}$

Where n is the refractive index of the material of the lens 102 and α isthe base angle of the multiple uniform triangular prisms that form themicroprism array 110. Since the microprism array 110 can includemultiple uniform triangular prisms. By way of an example, the base angleα can be 45° and the deviation angle δ to be 24.6349° based on thereflective index of Acrylic glass, which is n=1.491.

An image processing unit 130 can be electrically and physically coupledto the image capture function 106. The image processing unit 130 canreceive the output of the image capture function 106 including thecoordinates of the left image 124 and the right image 126. The imageprocessing unit 130 can parse the output of the image capture function106 and coordinates in order to produce a three dimensional image of thetarget object 114. The image processing unit 130 can be coupled to anillumination source 132, such as a light emitting diode (LED) lightsource, which can be controlled to illuminate the target object 114 whenthere is insufficient illumination to capture the left image 124 and theright image 126. The illumination source 132 can be an Infrared source,coupled to the image processing unit 130 for activating the imagecapture function 106 having an Infrared mode of operation. Theillumination source 132 when operating as the Infrared source willprovide better segmentation of the left image 124 and the right image126.

It has been discovered that the three dimensional image capture system100 can provide a three dimensional image from a two dimensional camerastructure at a significantly reduced cost from a stereo camera set-up.By applying the microprism array 110 spaced away from the lens 102, theleft image 124 and the right image 126 can be analyzed for forming thethree dimensional image of the target object 114. The image processingunit 130 of the three dimensional image capture system 100 canpredictably produce the left image 124 and the right image 126 forprocessing of the three dimensional image.

Referring now to FIG. 2, therein is shown an architecture diagram of athree dimensional image capture system 200 with image conversionmechanism in an example implementation. The architecture diagram of thealternative embodiment of the three dimensional image capture system 200depicts the microprism array 110 spaced a lens adapter length 202 fromthe image capture function 106. The lens adapter length 202 can be equalto the focal distance (f_(x)) 108 of FIG. 1 plus the array distance(U_(Z)) 112 of FIG. 1.

A field of view 204 can bracket the left virtual object 118 and theright virtual object 120 and the target object 114. The field of view204 represents the widest angle that can be captured by the threedimensional image capture system 100. In order to provide sufficientinformation to construct a three dimensional picture of the targetobject 114, both of the left virtual object 118 and the right virtualobject 120 must be within the field of view 204. The construction of thethree dimensional picture of the target object 114 represents aforeground process provides objects at a greater distance in a twodimensional background. It is understood that objects close to a viewercan be dimensionally detailed while more distant objects have lessidentifiable depth and are relegated to the background.

As an example an ideal pinhole camera placed at an optical center 206will have the following characteristics. The Z axis can be along theoptical axis 104 of the three dimensional image capture system 100. Thebase plane of the microprism array 110 is parallel to the image plane ofthe image capture function 106. The microprism array 110, can be made upof uniform triangular prisms, with base angle α can be placed at thearray distance (U_(Z)) 112 from the image capture function 106.

The target object 114 can be designated by K at location [x_(p), y_(p),z_(p)], two virtual object points can be horizontally displaced by themicroprism array 110, K_(l) at location [x_(pl), y_(p), z_(p)] and K_(r)at location [x_(pr), y_(p), z_(p)]. The horizontal displacement of theleft virtual object 118 and the right virtual object 120 is shown by thedifference in the displacement in the x-axis with no change in they-axis or z-axis displacement. The virtual object separation distance(D_(px)) 122 along the X axis between the left virtual object 118 andthe right virtual object 120 can be given by:D _(px) =x _(pr) −x _(pl)=2·z _(p)·tan(δ)  Equation 4

The pinhole camera of the example, with optical center 206 at O,captures the left image 124 and the right image 126 of the left virtualobject 118 and the right virtual object 120, K_(l) and K_(r). A width208 of the image capture function 106 (along X axis) can be W and aheight (along Y axis) can be H, both in pixel units. The field of view204 displayed horizontally, can be considered the angular aperture ofthe three dimensional image capture system 100 is ϕ. The horizontalfocal length of the camera can be given by:

$\begin{matrix}{f_{x} = \frac{w}{2 \cdot {\tan\left( {\phi/2} \right)}}} & {{Equation}\mspace{14mu} 5}\end{matrix}$

The left virtual object 118 and the right virtual object 120 can belocated at the picture depth z_(p) according to the co-ordinate system.The horizontal disparity in pixel units (along X axis) between the leftimage 124 and the right image 126, I_(l) and I_(r) can be given by:

$\begin{matrix}{d_{px} = {\frac{D_{px}}{z} \cdot \frac{\left( {W/2} \right)}{\tan\left( {\phi/2} \right)}}} & {{Equation}\mspace{14mu} 6}\end{matrix}$

The picture depth (z_(p)) 116 of the target object 114 from the opticalcenter 206 can be equal to z_(p)+u_(z)+f_(x). Substituting values ofD_(px) and f_(x) in Equation 5, we obtain the relation betweenhorizontal disparity in pixel units in the left image 124 and the rightimage 126 and the depth of the object in the co-ordinate system can becalculated by:

$\begin{matrix}{d_{px} = {\frac{2 \cdot z_{p} \cdot {\tan(\delta)}}{z_{p} + u_{z} + f_{x}} \cdot \frac{\left( {W/2} \right)}{\tan\left( {\phi/2} \right)}}} & {{Equation}\mspace{14mu} 7}\end{matrix}$

It has been discovered that the combination of the microprism array 110and the image capture function 106 can be a less expensive and lesscumbersome solution that is equivalent to a traditional stereo systemconsisting of two cameras placed at O separated by a baseline distance Bgiven by:B=2·u _(z)·tan(δ)  Equation 8

Referring now to FIG. 3, therein is shown an operational block diagramof an image capture process 301. The operational block diagram of theimage capture process 301 can include an N^(th) microprism 302 and aN+1^(st) microprism 304, of the microprism array 110,vertically-aligned. The N^(th) microprism 302 and the N+1^(st)microprism 304 can each be the uniform triangular prism, such as atriangular prism, having equal base angles and a prism vertex 305 withvertical and parallel alignment.

By way of an example, the N^(th) microprism 302 and the N+1^(st)microprism 304 can be right triangular prisms having their equal baseangles emanate from a common plane. The N^(th) microprism 302 and theN+1^(st) microprism 304 are abutted in order to form the microprismarray 110. It is understood that the number N can be any positiveinteger.

The N^(th) microprism 302 and the N+1^(st) microprism 304 can each haveequal base angles 306. A light source 308, such as a parallel lightsource or a laser, spaced at an extreme distance can provide light rays310 perpendicular to a microprism base plane 312.

The microprism array 110 can be made up of the N^(th) microprism 302 andthe N+1^(st) microprism 304, such as dispersing prisms, with the prismvertex 305 vertically aligned for each of the microprisms 302 in themicroprism array 110. The light rays 310 entering the N^(th) microprism302 and the N+1^(st) microprism 304, will emerge having been deflectedhorizontally from their original direction, by an angle δ known as adeviation angle (δ) 314. The smallest value of the deviation angle (δ)314 can be the ‘minimum deviation’, δ_(m). A uniform triangular prismhas identical values of the base angle 306, which we denote by α. If therefractive index of the material of the prism can be n, the relationbetween minimum deviation angle δ and base angle α can be calculated bya processor, such as an embedded control processing unit, an arrayprocessor, numerical control processor, or a combinational logicprocessor.

The deviation angle (δ) 314 can be calculated by:

$\begin{matrix}{\delta_{m} = {{2{\sin^{- 1}\left( {n\;{\sin\left( \frac{\alpha}{2} \right)}} \right)}} - \alpha}} & {{Equation}\mspace{14mu} 3}\end{matrix}$

Where n is the refractive index of the material of the lens 102 of FIG.1 and a is the base angle of the multiple uniform triangular prisms thatform the microprism array 110. Since the microprism array 110 caninclude multiple of the uniform triangular prisms. The base angle (α)306 can be equal to 45° and the deviation angle (δ) 314 can be equal to24.6349° based on the reflective index of Acrylic glass, which isn=1.491.

It has been discovered that the fixed value of the deviation angle (δ)314 can provide a standard separation of the left image 124 and theright image 126 on the image capture function 106. The standardseparation of the left image 124 and the right image 126 can aid in theidentification and correlation of matching points for producing thethree dimensional image. This reduces the circuitry required to producethe three dimensional image, while providing a detailed threedimensional display.

Referring now to FIG. 4, therein is shown a hardware block diagram ofthe three dimensional image capture system 100. The hardware blockdiagram of the three dimensional image capture system 100 depicts anoptical assembly 402 including the microprism array 110 and an opticsfunction 404, having the lens 102 of FIG. 1 and the image capturefunction 106 of FIG. 1.

The optics function 404 can be coupled to a video acquisition function406 for transferring a video data 408 from the image capture function106. The video data 408 can be a stream of pixel data for displaying theleft image 124 of FIG. 1 and the right image 126 of FIG. 1. The videoacquisition function 406 can be a memory or register structure forreceiving and ordering the video data 408. The video acquisitionfunction 406 can perform some amount of pre-processing of the video data408.

The video acquisition function 406 can be coupled to a video processor410. The video processor 410 can perform initialization and calibrationprocesses for the video acquisition function 406. The video processor410 can also identify corresponding points in the left image 124 and theright image 126, in preparation for determining a depthmap (not shown)for the target object 114 of FIG. 1.

The video processor 410 can search the video data 408 for matchingcorrespondence points in the left image 124 and the right image 126. Thesearch for the correspondence points can be performed in a horizontalregion across the field of view 204 of FIG. 2. Since the videoacquisition function 406 can provide a search buffer as well as areceiving buffer, the video processor 410 can identify all of thecorrespondence points in the search frame while the receiving frame canbe being loaded with the video data 408.

The video processor 410 can be coupled to a display application function412, through a depthmap bus 414 for transferring the depthmap derivedfrom the left image 124 and the right image 126, for displaying a threedimensional image of the target object 114. The video processor 410 canperform initialization and maintenance functions in the displayapplication function 412. The depthmap bus 414 can convey the depthmapcalculated by the video processor 410 and any background video data forthe image frame. The depthmap bus 414 can be implemented as a parallelbus, a serial link, or a combination thereof.

The display application function 412 can assemble the three dimensionalrendering of the target object 114 for display, transfer, or acombination thereof. The display application function 412 can onlyrender the three dimensional view of the target object 114 if both theleft virtual object 118 of FIG. 1 and the right virtual object 120 ofFIG. 1 are within the field of view 204 of FIG. 2. A depth sensingregion can be defined based on the distance of the target object 114from the microprism array 110, the field of view 204 of FIG. 2, and thedeviation angle (δ) 314 of FIG. 3. The depth sensing region can becalculated by:

$\begin{matrix}{Z_{pm} = \frac{u_{z} + f_{x}}{\left( {\frac{\tan(\delta)}{\tan\left( {\phi/2} \right)} - 1} \right)}} & {{Equation}\mspace{14mu} 9}\end{matrix}$

Where U_(z) can be the array distance (U_(Z)) 112, f_(x) can be thehorizontal focal length of the three dimensional image capture system100 from equation 5, δ can be the deviation angle (δ) 314 from equation3, and ϕ can be the field of view 204. The depth sensing region Z_(pm)can extend from the microprism array 110 and reduce linearly as afunction of the deviation angle (δ) 314 and the distance from themicroprism array 110 to the target object 114 of FIG. 1. Any of thevideo data 408 that falls outside the depth sensing region can betransferred as background data from the video processor 410 to thedisplay application function 412 for rendering the three dimensionalimage of the video frame.

It has been discovered that the three dimensional image capture system100 provides a method and apparatus for developing three dimensionalimages from hardware that was developed for two dimensionalapplications. The application of the microprism array 110 can generatethe left image 124 and the right image 126 for a single instance of thetarget object 114 and a single instance of the lens 102 of FIG. 1.

The three dimensional image capture system 100 has been described withfunction functions or order as an example. The three dimensional imagecapture system 100 can partition the functions differently or order thefunctions differently. For example, the video processor 410 could coupledirectly to the optics function 404 without the buffering capability ofthe video acquisition function 406.

The functions described in this application can be hardwareimplementation or hardware accelerators in the video processor 410, thevideo acquisition function 406, or in the display application function412. The functions can also be hardware implementation or hardwareaccelerators within the three dimensional image capture system 100 butoutside of the video processor 410.

Referring now to FIG. 5, therein is shown a flow chart of a method 500of the three dimensional image capture system 100 of FIG. 1 in adepthmap generation. The flow chart of the method 500 depicts a videoinput function 502 for receiving the video data 408 of FIG. 4. The videodata 408 can be used in two threads by the video processor 410 of FIG.4. The video data 408 can be concurrently transferred to a framecoherence function 504 and a correspondence points function 506. Theframe coherence function 504 can be a hardware accelerator forcalculating the frame to frame coherence of the objects in successiveframes by comparing the position, as noted by the coordinates [x_(p),y_(p), z_(p)] of objects in the successive frames. The correspondencepoints function 506 can be a hardware accelerator for parsing the videodata in search of correspondence points in the left image 124 and theright image 126. The correspondence points function 506 can perform ahorizontal search for a corresponding point 507 in both the left image124 and the right image 126, which can include, pixel block matching,regional feature matching, or a combination thereof.

The correspondence points function 506 can pass the corresponding points507 to a compute disparity function 508 for processing. The computedisparity function 508 can calculate the horizontal distance between thecorresponding points on a pixel by pixel basis by applying equation 1.The horizontal disparity of the corresponding points can be compiledinto a disparity map 509, such as an array containing the disparityvalues of each of the pixels, used to calculate the absolute depth ofthe corresponding points 507. The disparity map 509 can compile all ofthe disparity d_(px) of the pixel regions in a video frame by applyingequation 6, as shown above.

The compute disparity function 508 can be coupled to a compute depthmapfunction 510 for calculating an absolute value of a depthmap 512 basedon the disparity map 509 from the compute disparity function 508. Theinitial calculation of the depthmap 512 can be biased by theinstantaneous data provided from the video processor 410 in the computedisparity function 508.

In order to compensate for any error induced between frames of the videodata 408, a temporal coherence correction function 514 can receive anadjusted disparity 505 from the frame coherence function 504 and thedepthmap 512 from the compute depthmap function 510. The depthmap 512can be adjusted to maintain the frame to frame coherence and continuityof the three dimensional image provided as the output of the temporalcoherence correction function 514. The temporal coherence correction canbe performed by applying the adjusted disparity 505, based on the frameto frame changes, as calculated by:d _(t) ′=d _(t) +αs(d _(t-1) −d _(t))  Equation 10

Where the d_(t)′ is the frame adjusted disparity 505, d_(t) is thedisparity calculated for the current frame, and αs(d_(t-1)−d_(t)) is theframe-to-frame difference in disparity, between the current frame andthe previous frame, adjusted by a scaling factor as. The scaling factorincludes s, which is inversely proportional to the motion vector betweenthe two frames. The scaling factor also includes α, which is a heuristicweight that can be adjusted to indicate the importance of the temporalcoherence.

It has been discovered that the three dimensional image capture system100 can provide the three dimensional image based on a single instanceof the lens 102 of FIG. 1 and the microprism array 110 of FIG. 1. Thevideo processor 410 can identify correspondence points by thecorrespondence points function 506, generate the depthmap 512, andadjust it to maintain coherence from frame to frame by the temporalcoherence correction function 514. The three dimensional image outputfrom the temporal coherence correction function 514 can be a singleframe or a video stream of frames with a three dimensional image of thetarget object 114.

The method 500 includes: illuminating an image capture functionconfigured to generate video data in a block 502; focusing a left imageand a right image, through a lens, on the image capture function in ablock 506; horizontally deflecting the left image and the right imagefrom a microprism array in a block 508; and calculating a depthmap fromthe left image and the right image in the video data in a block 510.

Referring now to FIG. 6, therein is shown an example of a video frame601 as parsed by the correspondence points function 506 of FIG. 5. Thearchitectural diagram of the video frame 601 depicts pixel regions 602throughout the video frame 601. The video processor 410 of FIG. 4 canidentify a template block 604, located at a column n 606 and a row m608, as containing a prominent feature of the left image 124 of FIG. 1.The video processor 410 must then identify a correspondence point forthe right image 126 of FIG. 1 by a block matching process within the rowm 608.

Based on the deviation angle (δ) 314 of FIG. 3, an exclusion area 610can be imposed on both sides of the template block 604 within the row m608. The exclusion area 610 can prevent false correspondence matchesfrom occurring within the left image 124. A search for a correspondingblock can be performed within a target region 612. The search utilizes ablock matching process, having Sum of Squared Difference block matchingbased on Fast Fourier Transform, to identify a correspondence point 614within the row m 608.

It is understood that the search for the corresponding block can belimited to the horizontal row m 608, because the N^(th) microprism 302of FIG. 3 and the N+1^(st) microprism 304 of FIG. 3, each having theprism vertex 305 of FIG. 3 vertically aligned, so the deviation angle(δ) 314 of FIG. 3 is only seen in the horizontal direction. The minimumvalue of the deviation angle (δ) 314 can determine the exclusion area610 because each point the left image 124 or the right image 126 will bedisplaced at least two times the minimum value of the deviation angle(δ) 314.

It has been discovered that the search for the corresponding block canbe performed only in the row m 608 because the microprism array 110 onlydisplaces the left image 124 and the right image 126 horizontally due tothe prism vertex 305 of the N^(th) microprism 302 and the N+1^(st)microprism 304 being vertically aligned and parallel. Any featureidentified in the left image 124 will have a corresponding featurewithin the same row for the right image 126. This allows the videoprocessor 410 to minimize the search time for the correspondence point614.

Referring now to FIG. 7, therein is shown an example of a video frame701 as manipulated by the compute disparity function 508 of FIG. 5. Thearchitectural diagram of the video frame 701 depicts the template block604 having been identified by the video processor 410 of FIG. 4 and thecorrespondence point 614. Once matching pairs of the template block 604and the correspondence point 614 are found for all of the pixel regions602 corresponding to the left image 124 and the right image 126, thevideo processor 410 can compute the pixel disparity for these pairsusing their locations in the video frame 701.

During the execution of the correspondence points function 506 thetemplate block 604 can have a match with the corresponding point 614,but the corresponding point 614 can have a stronger match with an offsetblock 702. The compute disparity function 508 can calculate a disparityfor the three blocks that represents the mean value of the disparitybetween the template block 604 and the corresponding point 614 averagedwith the disparity between the corresponding point 614 and the offsetblock 702. The same disparity value can be assigned to each of theblocks.

The compute disparity function 508 can assign a disparity for all of thepixel regions 602 in the video frame 701. Once all of the pixel regions602 for the video frame 701 have been calculated, a median filter can beapplied to the disparity data, of the computed disparity map, in orderto impose smoothness constraints. The median filter can adjust thedisparity of each of the pixel region 602 in the video frame 701 byaveraging the disparity value of adjacent pixel regions 602.

Referring now to FIG. 8, therein is shown a flow of the compute depthmapfunction 510. The flow diagram of the compute depthmap function 510depicts a receive disparity data function 802, in which the disparitydata from all of the pixel regions 602 of FIG. 6 in the video frame 701of FIG. 7 can be tabulated. A relation between the real depth z_(p) ofan object point and the disparity d_(px) between the left image 124 ofFIG. 1 and the right image 126 of FIG. 1 corresponding to the targetobject 114 of FIG. 1 is given by Equation 7 as:

$\begin{matrix}{d_{px} = {\frac{2 \cdot z_{p} \cdot {\tan(\delta)}}{z_{p} + u_{z} + f_{x}} \cdot \frac{\left( {W/2} \right)}{\tan\left( {\phi/2} \right)}}} & {{Equation}\mspace{14mu} 7}\end{matrix}$

The substitute known values function 804 can replace the known physicalvalues for the set-up of the three dimensional image capture system 100of FIG. 1. The width 208 of FIG. 2 of the image capture function 106 ofFIG. 1 can be W The horizontal field of view 204 of FIG. 2, or angularaperture of the three dimensional image capture system 100 can be ϕ. Thefocal distance (f_(x)) 108 of FIG. 1 can be a constant for a singleinstance of the video frame 701 of FIG. 7. The array distance (U_(Z))112 of FIG. 1 is known and fixed. The deviation angle (δ) 314 of FIG. 3is fixed by design. It is understood that the video processor 410 ofFIG. 4 can control and detect the focal distance (f_(x)) 108 and thearray distance (U_(Z)) 112 as part of an autofocus routine during theoperation of the three dimensional image capture system 100.

A generate depth map function 806 can generate the picture depth (z_(p))116 of FIG. 1 value for each of the pixel regions 602 given the list ofconstants for a particular design and the disparity d_(px) values fromthe disparity map 509 of FIG. 5. The generate depth map function 806 cancalculate the picture depth Z_(p) 116 of FIG. 1 value for each of thepixel regions 602 to generate the depthmap 512 of FIG. 5.

The resulting method, process, apparatus, device, product, and/or systemis straightforward, cost-effective, uncomplicated, highly versatile,accurate, sensitive, and effective, and can be implemented by adaptingknown components for ready, efficient, and economical manufacturing,application, and utilization. Another important aspect of an embodimentof the present invention is that it valuably supports and services thehistorical trend of reducing costs, simplifying systems, and increasingperformance.

These and other valuable aspects of an embodiment of the presentinvention consequently further the state of the technology to at leastthe next level.

While the invention has been described in conjunction with a specificbest mode, it is to be understood that many alternatives, modifications,and variations will be apparent to those skilled in the art in light ofthe aforegoing description. Accordingly, it is intended to embrace allsuch alternatives, modifications, and variations that fall within thescope of the included claims. All matters set forth herein or shown inthe accompanying drawings are to be interpreted in an illustrative andnon-limiting sense.

What is claimed is:
 1. A three dimensional image capture systemcomprising: an image capture device configured to generate video data; alens, coupled to the image capture device, configured to focus a leftimage and a right image; a microprism array, coupled to the lens,configured to horizontally deflect the left image and the right image;and an image processing unit, coupled to the image capture device,configured to calculate a depthmap from the left image and the rightimage, captured simultaneously, in the video data including identifyingthe video data that falls outside a depth sensing region as backgrounddata, wherein the depth sensing region extends from the microprism arrayand reduces linearly based on a deviation angle, of the microprismarray, and a distance from the microprism array to a target object. 2.The system as claimed in claim 1 wherein the image processing unitincludes a video processor configured to calculate the depthmap.
 3. Thesystem as claimed in claim 1 wherein the microprism array includesmultiple vertically-aligned uniform triangular prisms configured todeflect a left virtual object and a right virtual object.
 4. The systemas claimed in claim 1 wherein the image processing unit includes a videoacquisition function configured to buffer the video data for identifyinga correspondence point.
 5. The system as claimed in claim 1 furthercomprising an illumination source, coupled to the image processing unit,configured to illuminate the target object.
 6. The system as claimed inclaim 1 wherein: the image capture device has a width centered on anoptical axis of the lens; the lens is spaced a focal distance (fx) fromthe image capture device; the microprism array is spaced an arraydistance (U_(Z)) from the lens opposite the focal distance (fx); and theimage processing unit is configured to calculate a disparity map fromthe left image and the right image.
 7. The system as claimed in claim 1wherein the image processing unit includes a video processor configuredto calculate a disparity map from a video acquisition function and tocalculate the depthmap for a display application function.
 8. The systemas claimed in claim 1 wherein the microprism array includes multiplevertically-aligned uniform triangular prisms having equal base angles.9. The system as claimed in claim 1 wherein the image processing unitincludes a video acquisition function that provides a memory structureconfigured to buffer the video data for identifying a disparity map. 10.The system as claimed in claim 1 further comprising an illuminationsource coupled to the image processing unit configured to illuminate thetarget object to generate a left virtual object and a right virtualobject from the microprism array.
 11. A method to capture threedimensional images comprising: illuminating an image capture functionconfigured to generate video data; focusing a left image and a rightimage, through a lens, on the image capture function; horizontallydeflecting the left image and the right image from a microprism array;and calculating a depthmap from the left image and the right image,captured simultaneously, in the video data including identifying thevideo data that falls outside a depth sensing region as background data,wherein the depth sensing region extends from the microprism array andreduces linearly based on a deviation angle, of the microprism array,and a distance from the microprism array to a target object.
 12. Themethod as claimed in claim 11 wherein calculating the depthmap from theleft image and the right image includes parsing a disparity map.
 13. Themethod as claimed in claim 11 further comprising horizontally deflectinga left virtual object and a right virtual object for forming the leftimage and the right image.
 14. The method as claimed in claim 11 whereincalculating the depthmap includes buffering the video data foridentifying a correspondence point.
 15. The method as claimed in claim11 further comprising illuminating the target object for horizontallydeflecting the left image and the right image.
 16. The method as claimedin claim 11 further comprising: centering a width of the image capturefunction on an optical axis of the lens; determining a focal distance(fx) between the lens and the image capture function; recording an arraydistance (U_(Z)) between the lens and the microprism array; andcalculating a disparity map from the left image and the right image. 17.The method as claimed in claim 11 further comprising calculating adisparity map and calculating the depthmap for a display applicationfunction.
 18. The method as claimed in claim 11 wherein horizontallydeflecting the left image and the right image includes illuminatingmultiple vertically-aligned uniform triangular prisms having equal baseangles configured to generate a deflection angle.
 19. The method asclaimed in claim 11 wherein calculating the depthmap includescalculating a disparity map while buffering the video data.
 20. Themethod as claimed in claim 11 further comprising illuminating the targetobject for horizontally deflecting the left image and the right imageincludes reflecting a left virtual object and a right virtual objectfrom the microprism array.